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ABSTRACT 

This document presents information on the Third International 
Mathematics and Science Study (TIMSS), briefly reviews how the study was 
conducted, and summarizes the findings concerning student achievement. The 
policies and practices that appear to enable high achievement in math and 
science and that suggest important policy direction for the U.S. are 
discussed. (KHR) 
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WhatisTIMSS? 

The Third International Mathematics and Science 
Study (TIMSS) is the most comprehensive and most 
closely monitored international study of math and sci- 
ence achievement to date. TIMSS tells us about educa- 
tion systems and the expectations of countries regarding 
all of their students and the best of their students. It is 
not about individual students, teachers, or schools. The 
American Federation of Teachers (AFT) believes that the 
TIMSS findings provide valuable information that can 
be used to improve the quality of American education. 

We begin this brief with a review of how the study 
was conducted and a summary of the findings about 
student achievement. These findings have received the 
most attention and provide educators with important 
data on student performance. The bulk of this brief, 
however, concerns the less reported and most important 
findings of TIMSS: the policies and practices that 
appear to enable high achievement in math and sci- 
ence — and which suggest important policy direction for 
the U.S. Among other things, TIMSS suggests that a 
policy push that focuses narrowly on “teacher perfor- 
mance” and not on issues such as standards, curriculum, 
and teacher development is not likely to succeed. 

The study sampled student achievement at three 
levels: 



1. 9-year-olds (U.S. fourth grade) 

2. 13-year-olds (U.S. eighth grade), and 

3. students in their final year of secondary school 
(U.S. 12th grade). 

Twenty-six countries participated at the fourth-grade 
level, 41 countries at the eighth-grade level, and 21 
countries in the final year of secondary school. There 
were two goals for the final year of secondary school. 
One was to find out what students knew when they 
exited the basic education system, whether they were 
entering the work force or university. The other was to 
examine the mathematical knowledge of the top 10 
percent to 20 percent of students in each country who 
take advanced math courses. Sixteen countries assessed 
“advanced math” students. U.S. advanced math students 
include those who took either calculus or pre-calculus. 

All participating countries were required to adminis- 
ter the assessment at the 13-year-old level. The other age 
levels were optional. In addition to surveying student 
achievement, TIMSS gathered extensive information 
about education systems, curricula, and instruction. It 
also conducted case studies of the U.S., Japan, and 
Germany. 

Particular attention was focused on eighth-grade 
mathematics, including a ground-breaking videotape 
study of classroom instruction in Germany, Japan, and 
the U.S. New findings continue to be released as more 
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analyses of the data are completed. 

Criticism regarding the administration of previous 
international studies led to great care being taken to 
make sure that TIMSS samples were representative of 
each country. More than any previous international 
study, these findings are being used in many school 
communities to examine policies and practices related 
to mathematics instruction. It is, therefore, important 
to examine how these rich findings inform AFT goals 
for improving education and, in particular, how the 
TIMSS results can inform the current U.S. debate con- 
cerning whether mathematics instruction in the U.S. 
should go “back to the basics” again or continue with 
some kind of reform. 

What Did TIMSS Find? 

TIMSS examined the achievement data from a 
number of viewpoints including: 

1 . Average scores for each country’s students. This 
analysis provides a picture of typical student achieve- 
ment. 

2. The range of scores for each country. Differences 
between each country’s top and bottom students give 
an indication of how well all students are being edu- 
cated. 

3. The top students in each country. This is an analysis 
of the top 10 percent of students in the TIMSS sam- 
ple. If all countries were equal, each would have 10 
percent of its students in this group. Alas, this is not 
the case. 



Table 1 

Percent of TIMMS countries that significantly 
outperform the U.S. 

80 




I I 



4th grade 8th grade 1 2th grade 

Sources: 

(Fourth Grade): Pursuing Excellence: A Study of U.S. Fourth-fade 
Mathematics and Science Achievement in International Contdjk 997), U.S. 
Department of Education, National Center for Education Statistics. 

(Eighth Grade): The TIMSSVideotape Classroom Study: Methods and 
Findings from an Exploratory Research project on Eigh tirade Mathematics 
Instruction in Germany, Japan, and the £7.5(1999), U.S. Department of 
Education, National Center for Education Statistics. 

(12th Grade): Pursuing Excellence : A Study of U.S. 12th-Gde Mathematics 
and Science Achievement in International Co/7fe(i£998) , U.S. Department 
of Education, National Center for Education Statistics. 

Average Student Achievement 

The picture of U.S. achievement in mathematics is 
not a happy one. The longer students are in school, the 
worse their achievement is relative to students in other 
countries (see Table 1). The chart shows the increasing 
percentage of participating countries whose students 
outperform U.S. students. 



Table 2 

Eighth-grade Math Score Distribution 



FRANCE 



USA 



200 250 




700 750 800 



Source: Mathematics Achievement in the Middffears: IE As Third International Mathematics and Science Study (TZM££1996), Beaton, A.E., Mullis, I.V.S., 
Martin, M.O., Gonzalez, E.J., Kelly, D.L., and Smith, T.A. 
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In addition to the downward slide when U.S. 
students are compared internationally as they move 
through school, TIMSS data reveal that: 

■ U.S. fourth-graders do as well as the average TIMSS 
fourth-grader in mathematics, with a mean score 
slightly above the international average. 

■ U.S. eighth-graders are significantly below the inter- 
national mean in math. 

■ U.S. 12th-grade achievement is among the lowest 
when compared to TIMSS students in other coun- 
tries who are in their final year of secondary school. 
On average, the general population of U.S. students 
outscored only Cyprus and South Africa. (It is 
important to note that this is so even though no 
Asian countries participated in this part of the study.) 

Range of Student Achievement 

As we have seen, U.S. students as a whole do not 
excel when compared to their international peers. What 
is even more disturbing is the wide gap that exists 
between our best and worst-achieving students, a gap 
we are striving to close. For example, at the eighth- 
grade level, the difference between students in the top 
and bottom 25 percent of those tested is 107 points for 
France and 128 points for the U.S. (See Table 2.) 

In addition, 63 percent of France s eighth-graders 
are included in the TIMSS top half, but only 45 per- 
cent of U.S. students are, an indication that something 
is working better for a greater percentage of students in 
France than in the U.S. The larger gap in the U.S. 
remains until the end of secondary school. The rest of 
the world (i.e., participating countries) has an average 
gap of 70 points between all their students and their 
advanced students in mathematics literacy. For the 
U.S., the gap is 90 points. 

TIMSS Top-Achieving Students 

When we compare the top U.S. students to the top 
students in other countries the picture varies little from 
the comparison of all students. 

■ Nine percent of U.S. fourth-graders would be includ- 
ed in a talent pool made up of the top 10 percent of 
all students who took TIMSS. Not bad. 

■ But only 5 percent of U.S. eighth-graders would be 
included in this pool instead of the expected 10 per- 







Table 3 






Comparison of All Students and Advanced Students in 


Mathematics and Science Literacy 




Mean Achievement 


Country 


Math and Science 
Literacy 


Mathematics 

Literacy 




All 

students 


Advanced 

math 

students 


All 

students 

students 


Advanced 

math 


Sweden 


555 (4.3) 


664(3.7) 


661(3.8) 


661(3.8) 


Switzerland 


531 (5.4) 


618(4.2) 


540(5.8) 


619(4.5) 


Denmark 


528 (3.2) 


594(2.9) 


547(3.3) 


613(3.0) 


Canada 


526 (2.6) 


587(3.7) 


519(2.8) 


588(3.3) 


Australia 


525 (9.5) 


604(8.1) 


522(9.3) 


606(7.6) 


Austria 


519 (5.4) 


567(5.9) 


518(5.3) 


564(6.1) 


Slovenia 


514 (8.2) 


531 (7.1) 


512 (8.3) 


530 (6.7) 


France 


505 (4.9) 


572 (5.0) 


523 (5.1) 


592(4.4) 


Germany 


496 (5.4) 


565 (4.1) 


495 (5.9) 


562 (4.4) 


Czech Rep. 


476 (10.5) 582 (7.2) 


466 (12.3) 


573 (7.8) 


Italy 


475 (5.3) 


521 (9.5) 


476 (5.5) 


519 (10.4) 


U.S. 


471 (3.1) 


554 (5.2) 


461 (3.2) 


551 (5.1) 


Cyprus 


447 (2.5) 


521 (6.1) 


446 (2.5) 


516 (6.5) 


INTERNATIONAL 

AVERAGE 


505 (1.6) 


575 (1.6) 


506 (1.7) 


576 (1.7) 


Source. Mathematics Achievement in the FinaYear of Secondary School: IEA 
Third International Mathematics and Science Study (TIMS$Jt 998) , 

Beaton, A.E., Mullis, I.V.S., Martin, M.O., Gonzalez, E.J., Kelly, D.L., 
and Smith, T.A. 



cent. Not good. 

■ And, the most advanced mathematics students in the 
United States (only about 5 percent of the 1 2th- 
grade cohort), performed similarly to 10 percent to 
20 percent of that same cohort in most other coun- 
tries. Terrible. 

■ To illustrate the point more graphically, when we 
look at only the advanced math students in each 
country, a student in France’s bottom quartile may 
score as well as a student in the U.S. top quartile. 
Half of the advanced students in the U.S. score 
below all French advanced students. (See Table 4.) 
Note the overlap just above the international average 
between France’s 25th percentile scores and the U.S. 
75th percentile scores. 
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Table 4 

Advanced Math Students Score Distribution 



International Mean = 501 
France Mean (557) 




200 250 300 350 400 450 500 550 600 650 700 750 



800 



United States (Mean 442) 

Key: | 

r~m~i 

Percentile 5 * 25 * 50 * 75 * 95 * 

Source: Mathematics Achievement in the FinaYear of Secondary School : IEA Third Internationa! Mathematics and Science Study (TIMSS). 998) , 
Beaton, A.E., Mullis, I.V.S., Martin, M.O., Gonzalez, E.J., Kelly, D.L., and Smith, T.A. 



What Does TIMSS Tell Us 
About Education Systems in 
Countries with High- 
Achieving Students? 

There are several things that most high-achieving 
TIMSS countries appear to have in common. But there 
is no one thing that can be interpreted as a miracle 
formula for high achievement. Several elements 
combine in each country to create a coherent system 
that works. Among the commonalities are! 

1 . A centrally defined set of goals and standards for 
what students should know. 

2. Teachers who are knowledgeable in the subjects they 
teach. Even in a country such as Singapore, where 
some primary-grade teachers may not complete four 
years of college, they /rcustpass rigorous subject-mat- 
ter exams at the end of secondary school in order to 
be accepted into teacher training. In contrast, 
approximately one-third of secondary math teachers 
in the U.S. do not have even a minor in the subject 
they are assigned to teach, and more than half of 
those teaching in inner-city schools are ill-prepared. 

3. Quality academic programs are in place to prepare 
those students who will not pursue postsecondary 
education. 



4. More focused curricula. Many TIMSS countries 
present fewer topics each year than do U.S. schools, 
thus allowing them to address topics in enough 
depth so that they do not have to be repeated every 
year. This enables them to move on to more complex 
applications and more rigorous content. 

5. More rigorous content. The level of content in the 
U.S. eighth grade equates to seventh-grade curricula 
elsewhere. 

6. No one pedagogy is used across all countries that do 
well. But, there are some common characteristics of 
mathematics instruction that is judged likely to lead 
to high achievement. The TIMSS findings concern- 
ing strong instructional lessons in math are summa- 
rized in the next section. 

7. Alignment of textbooks and curricula with standards 
and goals. 

8. Student accountability for reaching the standards 
and incentives for high achievement. Meeting stan- 
dards is key to entry into the most prestigious uni- 
versities and jobs. 
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What Have We Learned from 
TIMSS About Excellent 
Mathematics Instruction? 

One unique feature of TIMSS was a videotape study 
of classroom instruction. Dr. James Stigler 1 had eighth- 
grade mathematics lessons in the U.S., Germany, and 
Japan videotaped. Locations were selected to reflect the 
diverse settings in each country. Schools were the same 

1 For more information see, TheTeaching G3p(1999), Stigler, J. 
and Hiebert, J. The Free Press! New York, NY. 

Table 5 

Comparison of some lesson elements 
that correlate with strong lessons 




CONCEPTS EXPLICIT REASONING 

DEVELOPED LINKING 

I I U.S. 

■■ JAPAN 



ones in which the TIMSS student assessments were 
administered. Teachers and the class period to be video- 
taped were selected randomly. 

Analysis of the videotapes revealed a high correlation 
between strong lessons and the following practices. (See 
Table 5.) 

1. The goal of the lesson. Strong lessons were more 
likely to target mathematical thinking while weak 
ones were narrower in goal, seeking to teach students 
how to solve a particular kind of problem or carry 
out a specific procedure. 

2. Treatment of mathematical concepts. In strong 
lessons, the concepts were far more likely to have 
been developed rather than simply presented as rules. 

3. Multiple solution strategies. Strong lessons included 
multiple ways to solve problems. 

4. Strong lessons were focused and coherent. Weaker 
ones switched topics significantly more times. 

5. Strong lessons included mathematical reasoning. 
Whether this was formal proof or informal reason- 
ing, mathematical reasoning was explicit. 

6. Complexity. The complexity of tasks within a strong 
lesson was likely to increase. 

7. Type of problems. Strong lessons generally asked 
students to perform tasks that were not “routine.” 
That is, they weren’t plain calculation or problems 
for which students just applied a formula that was 
given to them. In strong lessons, students might have 
to figure out which formula to use or find more than 
one way to solve the problem. 

8. Connections. In strong lessons, teachers helped stu- 
dents make explicit connections between parts of the 
lesson to previous knowledge, and/or to statements 
and problems from earlier parts of the lesson. 

These practices are all supported by other research 
on how children learn and, particularly, on how they 
learn mathematics. 



Source : The TIMSSVideotape Classroom Study : Methods and Findings from an 
Exploratory Research project on EighthrfEle Mathematics Instruction in 
Germany, Japan, and the £7.5.(1999), U.S. Department of Education, 

National Center for Education Statistics. 
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What Does TIMSS Say about 
the Usual Recommendations 
To Boost U.S. Performance — 
Such as Longer School Days, 
More Homework, 
and More Basics? 

When talking about the performance of U.S. stu- 
dents, it is common for people to propose magical 
cures that take the form of more time in school, more 
homework, and going “back to the basics.” TIMSS 
data, however, show that these are not the solutions for 
improved math achievement. 

1. More time. U.S. students in the lower grades 
actually reported being in class for math and science 
for longer periods of time than their counterparts in 
other countries reported. Thus, the problem in the 
earlier years is not the time available or even the time 
scheduled. It is how that time is used and what level of 
mathematics is teug/tf However, in the final year of 
secondary school, advanced math and science 
students in the U.S. reported less class time for 
science and mathematics than their peers did. In 
many other countries students were taking two 
sciences, a practice that generally does not happen 
here. Many students in other countries reported 

more than five hours per week in math class. 

2. More homework. Eighth-grade students were asked 
how much homework or studying they did outside 
of school. Fourth- and eighth-grade students actually 
reported as much homework as their international 
peers did. 2 Twelfth-grade U.S. students reported less 
than other students in their final year of secondary 
school. The intellectual demands of homework are 
perhaps as great an issue as the amount of home- 
work. TIMSS found that U.S. teachers assign mostly 
routine tasks for homework, while Japanese home- 

2 The TIMSS Case Study on Japan indicates that a relatively small 
proportion of elementary students participate in the after-school 
juku schools and that they are more likely to be engaged in recre- 
ational or arts courses there. It is in eighth grade that more stu- 
dents begin attending for academic content to prepare for high 
school entrance exams at the end of grade 9. 



work places more intellectual demands on students. 

3. U.S. students are not falling behind in the basics. 
Rather, they falter most with multistep problems, 
mathematical reasoning, geometry, and solving prob- 
lems that are not routine . Indeed \ the curriculum 
analyses revealed that U.S. students repeat the basics as 
they mere through the grades rather than being exposec 
to higher levels of mathematicThis finding is con- 
firmed by the U.S. National Assessment of 
Educational Progress (NAEP). Students today out- 
perform their predecessors in basic items that have 
been administered every year since the early 1970s. 

Does TIMSS Shed 
Any Light on Pedagogy? 

The fact that there are high-achieving students who 
are taught using differing approaches tells us that no 
single approach can be called the right one. Japan 
believes that students must struggle with problems and 
find many different ways to solve them. French-speak- 
ing Belgium bases math instruction on set theory (the 
basis of the 1960 s New Math). In the Netherlands, 
math instruction is grounded in real life problems. The 
French require precision about definitions and lan- 
guage, but also ask that students be able to reason 
mathematically and draw conclusions from more than 
definitions. Students in all these countries outperform 
U.S. pupils. 

Researcher James Stigler says that his videotape 
study found that “in many ways Japanese teaching 
resembled the recommendations of the U.S. reform 
movement more closely than did American teaching,” 3 
but that “they re more teacher directed than you get the 
sense of from what the NCTM standards seem to 
recommend. These are lessons that are clearly highly 
controlled by the teacher.” 

What is clear! Mathematical reasoning is missing 
from the typical U.S. math lesson, and our content is 
less challenging than that of many other countries. It is 
also clear that U.S. students fare poorly trying to solve 
multistep word problems. They fare better on computa- 
tion problems with no context. To reach the U.S. goal 

3 Pursuing Excellence'. A study of U.S. Eighth-fade Mathematics and 
ScienceTeaching, Learning, Curriculum, and Achievement in 
International Contex({l996) . U.S. Department of Education, 
National Center for Education Statistics. 
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* of being first in the world in mathematics, we need to 
improve the balance of the curriculum devoted to basic, 
foundational skills and to more advanced and complex 
skills so that increasing numbers of students are pre- 
pared for the study of higher mathematics. But clearly, 
in this rebalancing, we must not lose a necessary 
focus — the need for children to master the basics. 

If we want to raise math achievement, teachers must 



understand the instructional practices that TIMSS 
found in strong math lessons. To help do this we must 
ensure that future and current teachers alike gain a level 
of mathematical knowledge that many, through no 
fault of their own, do not now have. Without that 
knowledge, it is difficult to successfully engage students 
in learning math concepts and procedures, mathemati- 
cal problem solving, and mathematical reasoning. 
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How Do TIMSS Results Support AFT s Reform Agenda? 



Goal 


TIMSS Finding 


1. Adopt rigorous academic 
standards. 


Our content level is below that of other countries. Most countries have 
nationally or state determined standards. 


2. Adopt curriculum aligned 
to those standards. 


U.S. curricula are unfocused and low-level. We teach more topics each 
year than 75 percent of TIMSS countries. U.S. teachers use mainly 
routine problems and tasks. 


3. Eliminate out-of-field 
teaching assignments. 


The concept of teaching out of field is unknown in most TIMSS 
countries. Teachers of math in other countries have been prepared to 
teach math. 


4. Improve the quality of teacher 
preparation, including the level of 
content knowledge required. This 
may include greater knowledge of 
the content Fields they teach and 
their most crucial ideas. 


Teachers in high-achieving countries are required to pass rigorous subject- 
matter exams before being admitted to teacher training. 


5. Align standards, curriculum, 
textbooks, and assessment. 


The education ministries in many countries must approve textbooks. 
Some publishers purposely develop them to match the learning goals. 
Even when there is no central adoption, curriculum goals are clear and 
centrally decided, making it easier for publishers to produce books that 
are aligned with the goals. 


6. Provide time for collaborative work, 
observation of good practice, and 
peer coaching. 


The value of collaboration on lessons and opportunities to observe 
colleagues teach is clear in the practices of Japan. 


7. Get rid of constant interruptions 
of lessons with which teachers 
are faced. 


Interruptions in U.S. classrooms exceed those of other countries. 
International educators were amazed by the interruptions and could not 
understand why they were permitted. 



For more information on TIMSS, contact: 

■ National Center for Education Statistics (NCES) 
http ://nces. ed . gov/ 1 imss 

■ International Association for the Evaluation of Educational Achievement (IEA) 
International Study Director, Albert Beaton, Boston College 
http://www.csteep.bc.edu/timss 

■ U.S. TIMSS Study 

Director, William Schmidt, Michigan State University 
http://ustimss.msu.edu 

■ American Federation of Teachers 
http://www.aft.org/timss 
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